Eliciting Knowledge from Pretrained Language Models for Prototypical Prompt Verbalizer

Recent advances on prompt-tuning cast few-shot classification tasks as a masked language modeling problem. By wrapping input into template and using verbalizer which constructs mapping between label space word space, can achieve excellent results in scenarios. However, typical needs manually designed requires domain expertise human efforts. And the insufficient may introduce considerable bias results. In this paper, we focus eliciting knowledge from pretrained models propose prototypical prompt for prompt-tuning. Labels are represented by embeddings feature rather than discrete words. The distances embedding at position of used criterion. To address problem random initialization parameters zero-shot settings, elicit to form initial embeddings. Our method optimizes contrastive learning. Extensive experimental several many-class text datasets with low-resource settings demonstrate effectiveness our approach compared other construction methods. implementation is https://github.com/Ydongd/prototypical-prompt-verbalizer .